unsloth

Author	SHA1	Message	Date
Daniel Han	4474e4bca4	Fix Gemma GGUF (#311 ) * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py * Fix inference * swiglu * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * fast inference * model * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update llama.py * Update utils.py * inference * Update llama.py * Update llama.py * Update llama.py * overhead * Update llama.py * Update llama.py * compile * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * lora mamtul * Update llama.py * Update llama.py * Update llama.py * offloaded checkpointing * Update llama.py * Update llama.py * Update _utils.py * Update _utils.py * Update _utils.py * Update llama.py * Update llama.py * Update gemma.py * Revert "Update gemma.py" This reverts commit `c68b59bbfd`. * Update _utils.py * Update _utils.py * Update _utils.py * Saving * sentencepiece_model_pb2 * Update llama.py * Update save.py * Update llama.py * padding side * Update tokenizer_utils.py * cache dir * Update tokenizer_utils.py * Update tokenizer_utils.py * Update pyproject.toml * Update pyproject.toml * Update tokenizer_utils.py * Update tokenizer_utils.py * Update llama.py * Update save.py * Update save.py * checkpoint * Gemma 1.1 * more models	2024-04-08 01:28:00 +10:00
Daniel Han	f3d05d19e3	Bug fixes (#308 ) * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py * Fix inference * swiglu * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * fast inference * model * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update llama.py * Update utils.py * inference * Update llama.py * Update llama.py * Update llama.py * overhead * Update llama.py * Update llama.py * compile * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * lora mamtul * Update llama.py * Update llama.py * Update llama.py * offloaded checkpointing * Update llama.py * Update llama.py * Update _utils.py * Update _utils.py * Update _utils.py * Update llama.py * Update llama.py * Update gemma.py * Revert "Update gemma.py" This reverts commit `c68b59bbfd`. * Update _utils.py * Update _utils.py * Update _utils.py * Saving * sentencepiece_model_pb2 * Update llama.py * Update save.py * Update llama.py * padding side * Update tokenizer_utils.py * cache dir * Update tokenizer_utils.py * Update tokenizer_utils.py * Update pyproject.toml * Update pyproject.toml * Update tokenizer_utils.py * Update tokenizer_utils.py * Update llama.py	2024-04-07 03:44:45 +10:00
Daniel Han	920f0ae6e5	Bug fixes (#306 ) * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py * Fix inference * swiglu * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * fast inference * model * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update llama.py * Update utils.py * inference * Update llama.py * Update llama.py * Update llama.py * overhead * Update llama.py * Update llama.py * compile * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * lora mamtul * Update llama.py * Update llama.py * Update llama.py * offloaded checkpointing * Update llama.py * Update llama.py * Update _utils.py * Update _utils.py * Update _utils.py * Update llama.py * Update llama.py * Update gemma.py * Revert "Update gemma.py" This reverts commit `c68b59bbfd`. * Update _utils.py * Update _utils.py * Update _utils.py * Saving * sentencepiece_model_pb2 * Update llama.py * Update save.py * Update llama.py * padding side	2024-04-06 04:31:24 +11:00
Daniel Han-Chen	db4a24e602	Gemma inference fix	2024-04-05 03:51:53 +11:00
Daniel Han-Chen	b122b76b26	Update gemma.py eabdullin	2024-04-04 22:46:32 +11:00
Daniel Han	7e1c6a62e2	Nightly (#299 ) * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py * Fix inference * swiglu * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * fast inference * model * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update llama.py * Update utils.py * inference * Update llama.py * Update llama.py * Update llama.py * overhead * Update llama.py * Update llama.py * compile * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * lora mamtul	2024-04-03 05:38:31 +11:00
Daniel Han	537577720c	Fix batched inference (#298 ) * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py * Fix inference * swiglu * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * fast inference * model * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update llama.py * Update utils.py * inference * Update llama.py * Update llama.py * Update llama.py * overhead * Update llama.py * Update llama.py * compile * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py	2024-04-03 04:56:11 +11:00
Daniel Han-Chen	8921867157	Revert "Temp fix batch inference (#294 )" This reverts commit `e209991ba1`.	2024-04-02 13:18:31 +11:00
Daniel Han	e209991ba1	Temp fix batch inference (#294 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit * Update mistral.py * Update mistral.py * Stats * Update mistral.py * attention_mask * Update llama.py * Update llama.py * batch * Temp fix batch inference * Update llama.py * Update gemma.py	2024-04-02 04:35:28 +11:00
Daniel Han	8e263b8b7d	Nightly (#293 ) Env checking	2024-04-01 04:38:12 +11:00
Daniel Han	74f79684da	Auto Healing Tokenizer (#283 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py * Patch tokenizer * Update chat_templates.py * Heal tokenizers * Update chat_templates.py * Update mapper.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update tokenizer_utils.py * Update chat_templates.py * tokenizer patching * patch_tokenizer * Update chat_templates.py * Update tokenizer_utils.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update tokenizer_utils.py * Edit	2024-03-28 04:16:50 +11:00
Daniel Han-Chen	bb45fdabb6	Update mapper.py	2024-03-24 14:00:04 +11:00
Daniel Han	b9d5ca53dc	lm_head issue (#266 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py * Update llama.py	2024-03-20 04:48:15 +11:00
Daniel Han	e269b0dc59	Fix Saving (#264 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * state_dict * Update save.py * whoami * Update llama.py * Update save.py	2024-03-19 19:55:02 +11:00
Daniel Han	696e8817ea	Fix GGUF and saving (#261 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code * lm_head * Update llama.py * save_pretrained_settings * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py	2024-03-19 04:32:28 +11:00
Daniel Han	36473e2d6e	Fix lm_head, embed_tokens (#258 ) * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py * Update __init__.py * Update fast_lora.py * dtype * Update llama.py * Update llama.py * Update llama.py * dtype * Update mistral.py * trust_remote_code	2024-03-18 04:18:15 +11:00
Daniel Han-Chen	16b3dd86b7	Update fast_lora.py	2024-03-17 22:46:44 +11:00
Daniel Han-Chen	4b9aaee6b1	Update fast_lora.py	2024-03-17 22:46:17 +11:00
Daniel Han-Chen	3685ec607d	Squashed commit of the following: commit `61d45c60db` Merge: `f024ce2` `64d847b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 22:12:44 2024 +1100 Merge branch 'main' into nightly commit `f024ce2821` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 22:12:18 2024 +1100 Update __init__.py commit `d38cf5387c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 22:09:23 2024 +1100 Update fast_lora.py commit `9c35a2c4b0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 22:08:33 2024 +1100 Update pyproject.toml commit `6edc35f686` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 20:18:00 2024 +1100 Update fast_lora.py commit `2a9d4fb947` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 20:10:30 2024 +1100 Bugs commit `14717be070` Merge: `5c24a3b` `c599ae0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 20:10:26 2024 +1100 Merge branch 'main' into nightly commit `5c24a3bc2e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 02:46:35 2024 +1100 Update README.md commit `fd729a7131` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 17 02:44:58 2024 +1100 Update README.md commit `7e9f092e9f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 23:17:35 2024 +1100 Update save.py commit `e3efca8778` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 22:36:57 2024 +1100 Update save.py commit `d58fa31e0c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 22:31:02 2024 +1100 Update save.py commit `64d954bd13` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 20:32:12 2024 +1100 Update save.py commit `815202f832` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 20:03:51 2024 +1100 GGUF commit `338b2c928b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 04:40:21 2024 +1100 Update README.md commit `f342425c1e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 04:34:51 2024 +1100 Update README.md commit `cef733a420` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 04:10:54 2024 +1100 Update fast_lora.py commit `e5bcab2a74` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 03:43:16 2024 +1100 Update fast_lora.py commit `80cfe132f6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 03:38:13 2024 +1100 Fix bugs commit `d8e98be90d` Merge: `51c2484` `39713e6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 16 00:14:02 2024 +1100 Merge branch 'main' into nightly commit `51c2484ffd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 22:35:44 2024 +1100 Update rope_embedding.py commit `3e93a78794` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 20:14:56 2024 +1100 Update rope_embedding.py commit `82d80e9e98` Merge: `718a6a1` `990c7a8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 19:50:20 2024 +1100 Merge branch 'main' into nightly commit `718a6a1c84` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 19:50:03 2024 +1100 Update pyproject.toml commit `385c6d44a8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 19:49:09 2024 +1100 Update pyproject.toml commit `9c9ede4680` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 19:48:39 2024 +1100 Update pyproject.toml commit `9d6c9c9ebc` Merge: `a12e4ea` `2c5c5bb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 05:07:09 2024 +1100 Merge branch 'main' into nightly commit `a12e4ea3e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 04:44:54 2024 +1100 Update chat_templates.py commit `c3e0e518d9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 04:30:03 2024 +1100 Update chat_templates.py commit `fe0e2d7baa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 03:50:32 2024 +1100 Update chat_templates.py commit `5dfe582c96` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 03:37:37 2024 +1100 Update chat_templates.py commit `ec73a776ec` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 15 03:34:32 2024 +1100 Update chat_templates.py commit `5c4241a240` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:27:20 2024 +1100 Update pyproject.toml commit `6841a303d9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:20:16 2024 +1100 Update pyproject.toml commit `341b5f46e6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:12:54 2024 +1100 Update pyproject.toml commit `6809846464` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:12:02 2024 +1100 Update pyproject.toml commit `77edfb10b7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:11:24 2024 +1100 Update pyproject.toml commit `0ed19ec170` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:10:10 2024 +1100 Update pyproject.toml commit `1f4c625f2a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:06:05 2024 +1100 Update pyproject.toml commit `8fa0aab61c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 20:04:01 2024 +1100 Update pyproject.toml commit `eb61377632` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 14 19:02:29 2024 +1100 Fix Colab commit `d6ac9b56c1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 12 20:12:55 2024 +1100 upcasting commit `b7c3190e97` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 12 00:46:57 2024 +1100 Update pyproject.toml commit `4d77c32425` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 23:28:03 2024 +1100 Update pyproject.toml commit `98573e8618` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 23:21:27 2024 +1100 kaggle new commit `4a794dd880` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 20:28:09 2024 +1100 Update pyproject.toml commit `a0c18c9880` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 20:05:06 2024 +1100 Update save.py commit `684eaae9ea` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 19:52:18 2024 +1100 GGUF incorrect commit `029d588603` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 19:13:42 2024 +1100 Update save.py commit `252a38a2df` Merge: `63b1f58` `3222377` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 19:13:29 2024 +1100 Merge branch 'main' into nightly commit `63b1f5879e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 04:05:43 2024 +1100 Update llama.py commit `cb0d937469` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 03:53:23 2024 +1100 Account for DoRA commit `2e87755a5c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 03:19:20 2024 +1100 Update llama.py commit `93d88ad68d` Merge: `aba595d` `8bea94c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 03:15:30 2024 +1100 Merge branch 'main' into nightly commit `aba595de9c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 03:07:44 2024 +1100 Update llama.py commit `133097d995` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 02:25:10 2024 +1100 Update save.py commit `b5d3d63df5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 02:15:33 2024 +1100 Update save.py commit `8be91b050f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 11 02:10:51 2024 +1100 Update chat_templates.py commit `23b7a5764c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 19:32:02 2024 +1100 Update fast_lora.py commit `c1728a9904` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 19:11:00 2024 +1100 Update fast_lora.py commit `c192ce3ed4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 19:09:10 2024 +1100 Update fast_lora.py commit `7e3abd19ba` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 18:36:35 2024 +1100 Update fast_lora.py commit `c1f3e70394` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 18:33:52 2024 +1100 Update fast_lora.py commit `08da057f04` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 18:09:58 2024 +1100 Update save.py commit `1e8922af2b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 17:42:41 2024 +1100 Revert commit `74fc5caa60` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 16:20:20 2024 +1100 Accuracy commit `35c6d776c4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 14:13:53 2024 +1100 Update save.py commit `6d2bc97117` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 14:08:38 2024 +1100 Update llama.py commit `baf8e4c0a8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 13:11:42 2024 +1100 Update llama.py commit `c0d9516255` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 13:10:03 2024 +1100 Update llama.py commit `91877f5506` Merge: `f887080` `1fcf9d4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 13:08:23 2024 +1100 Merge branch 'main' into nightly commit `f887080a6d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 04:31:40 2024 +1100 Tokenizer overwritten commit `1c1461ae09` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 04:08:02 2024 +1100 Update loader.py commit `14f063819a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 03:54:18 2024 +1100 model_name commit `daba749ee1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 03:48:05 2024 +1100 Update llama.py commit `457b7ba6d6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 02:57:41 2024 +1100 Update chat_templates.py commit `5c9629f5fe` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 10 02:55:07 2024 +1100 Update save.py commit `e6c3cdfc2d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 9 23:22:14 2024 +1100 Update save.py commit `f0a3c05b07` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 9 23:19:02 2024 +1100 Update save.py commit `58d1f1e03c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 9 20:05:52 2024 +1100 Update chat_templates.py commit `9b3dd3e9f8` Merge: `b321ada` `70f271b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 8 19:40:25 2024 +1100 Merge branch 'main' into nightly commit `b321adac2c` Merge: `03232df` `fedcafe` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 7 04:32:51 2024 +1100 Merge branch 'main' into nightly commit `03232dff4e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 7 02:48:15 2024 +1100 Update chat_templates.py commit `e67c6b4ec9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Mar 7 02:05:38 2024 +1100 Fix warning commit `5a7f52819e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 19:21:44 2024 +1100 Update rms_layernorm.py commit `837ba610cf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 18:18:07 2024 +1100 RoPE and Gemma precision commit `1e41fa0c8c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 05:07:37 2024 +1100 Update save.py commit `333d3d9a51` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 04:52:59 2024 +1100 Update gemma.py commit `85c052d8fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 04:48:26 2024 +1100 sqrt commit `ed1aa0007a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 04:24:10 2024 +1100 Update gemma.py commit `be81c07d43` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Mar 6 04:14:11 2024 +1100 Gemma precision commit `5a693a107d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 23:58:00 2024 +1100 Layernorms commit `e0a2463738` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 20:07:44 2024 +1100 Update pyproject.toml commit `160320b3d9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 18:55:50 2024 +1100 Update gemma.py commit `9f7f205f67` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 18:27:11 2024 +1100 Update rms_layernorm.py commit `43e710b064` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 18:21:59 2024 +1100 Fix Gemma merging commit `3137392ccf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 02:41:56 2024 +1100 Update gemma.py commit `fe35b127fc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 02:40:20 2024 +1100 Update gemma.py commit `5fae81b628` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 02:22:31 2024 +1100 Update gemma.py commit `961813b35f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Mar 5 01:45:08 2024 +1100 Update gemma.py commit `c027dacb18` Merge: `cb193f7` `7b7665d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 4 16:19:13 2024 +1100 Merge branch 'main' into nightly commit `cb193f7e0a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 4 16:14:50 2024 +1100 Update gemma.py commit `440c29273f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 4 16:11:12 2024 +1100 Update rms_layernorm.py commit `c31b27b7dc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 4 16:04:29 2024 +1100 Update rms_layernorm.py commit `6fa081ae77` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Mar 4 16:00:35 2024 +1100 Update rms_layernorm.py commit `aa2fb63048` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 19:32:18 2024 +1100 Update gemma.py commit `ac23e4b612` Merge: `1fea4ff` `fa2a43b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 19:29:41 2024 +1100 Merge branch 'main' into nightly commit `1fea4ffcf2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 18:07:55 2024 +1100 Update geglu.py commit `245fe4716c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 03:39:36 2024 +1100 Update _utils.py commit `e27523cd5e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 03:39:10 2024 +1100 Update __init__.py commit `65fbde6cd9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 03:37:43 2024 +1100 Update __init__.py commit `9a2e791e6c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 03:35:21 2024 +1100 Update llama.py commit `786885c38c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 03:17:59 2024 +1100 Approx gelu commit `1c7f0d21ee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Mar 3 02:24:39 2024 +1100 Update geglu.py commit `393e53b016` Merge: `c88ab10` `307f2da` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 2 18:29:02 2024 +1100 Merge branch 'main' into nightly commit `c88ab10a5c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 2 18:28:39 2024 +1100 Approx gelu commit `e032445694` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 2 02:39:11 2024 +1100 Update pyproject.toml commit `c970a2b3be` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Mar 2 02:38:40 2024 +1100 Small fixes commit `db87262625` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Mar 1 04:16:25 2024 +1100 Update pyproject.toml commit `d44bbf5f2e` Merge: `0866020` `d0c15bb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:17:47 2024 +1100 Merge branch 'nightly' of https://github.com/unslothai/unsloth into nightly commit `0866020037` Merge: `54b26c0` `2561964` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:17:12 2024 +1100 Merge branch 'main' into nightly commit `d0c15bb508` Author: Daniel Han <danielhanchen@gmail.com> Date: Thu Feb 29 00:17:03 2024 +1100 Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py commit `54b26c0466` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:16:19 2024 +1100 Update llama.py commit `074aa737ce` Merge: `b1892b5` `e7c53fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:14:42 2024 +1100 Merge branch 'main' into nightly commit `b1892b5511` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:09:00 2024 +1100 Update chat_templates.py commit `072dc0c447` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 29 00:02:40 2024 +1100 Update _utils.py commit `d37f284eaa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 28 23:59:49 2024 +1100 DoRA commit `967fed83c7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 28 22:02:18 2024 +1100 Update llama.py commit `a4cd4a6e41` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 27 01:40:45 2024 +1100 Update README.md commit `90a5c2c121` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 27 01:39:52 2024 +1100 Update README.md commit `4b7df80abd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 27 00:16:57 2024 +1100 Chat Templates commit `54ff6eb169` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 17:27:42 2024 +1100 Update cross_entropy_loss.py commit `7cf69dd9f2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 16:19:39 2024 +1100 Update cross_entropy_loss.py commit `1f450cd6ee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:44:22 2024 +1100 Update gemma.py commit `7a5db6758c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:39:43 2024 +1100 correct_dtype commit `cf16811ab9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:38:27 2024 +1100 Update gemma.py commit `637d01643c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:37:48 2024 +1100 Update llama.py commit `3d73560df3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:37:09 2024 +1100 Update llama.py commit `0d24acf0f5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:36:39 2024 +1100 Update llama.py commit `1aeca6f00c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:35:44 2024 +1100 RoPE commit `4ec1b216e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 14:15:37 2024 +1100 Update save.py commit `11c9ea4d1b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 05:06:04 2024 +1100 Update gemma.py commit `24e9d0af28` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 05:05:41 2024 +1100 Update gemma.py commit `0b693df646` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:37:12 2024 +1100 Update gemma.py commit `ae336e9663` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:36:10 2024 +1100 Update gemma.py commit `1d53b366a0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:32:49 2024 +1100 Update gemma.py commit `b1864710c5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:30:29 2024 +1100 Update gemma.py commit `686d9fa4c4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:27:55 2024 +1100 Update gemma.py commit `40169f9fd3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:23:38 2024 +1100 Update gemma.py commit `b0b38f770b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 04:20:19 2024 +1100 Update gemma.py commit `13e06a062b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:56:34 2024 +1100 Update gemma.py commit `fd5389a485` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:55:15 2024 +1100 Update cross_entropy_loss.py commit `a5abe39ded` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:53:49 2024 +1100 Update cross_entropy_loss.py commit `78c96ef83c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:50:34 2024 +1100 Update cross_entropy_loss.py commit `88cfe5eeb2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:48:16 2024 +1100 Update cross_entropy_loss.py commit `dba4c0355d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:46:18 2024 +1100 Update cross_entropy_loss.py commit `199460d5ef` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:43:22 2024 +1100 Update cross_entropy_loss.py commit `13718b9847` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:36:38 2024 +1100 Update cross_entropy_loss.py commit `48736a0755` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:35:45 2024 +1100 Update cross_entropy_loss.py commit `c1dbf6710e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:29:19 2024 +1100 Update cross_entropy_loss.py commit `b55d7ad4b2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:29:11 2024 +1100 Update cross_entropy_loss.py commit `bc3ff0f3a6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:28:22 2024 +1100 Update cross_entropy_loss.py commit `7bbce701c8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:26:42 2024 +1100 Update cross_entropy_loss.py commit `9c0b3cd431` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:19:26 2024 +1100 Update cross_entropy_loss.py commit `e696487e84` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:18:01 2024 +1100 Update cross_entropy_loss.py commit `c675220373` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:16:31 2024 +1100 Update cross_entropy_loss.py commit `bb6b6f238e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:15:19 2024 +1100 Update cross_entropy_loss.py commit `096544eff4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:14:04 2024 +1100 Update cross_entropy_loss.py commit `b5eff3bfbc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:12:23 2024 +1100 Update cross_entropy_loss.py commit `fdaa9493be` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 03:09:37 2024 +1100 Update cross_entropy_loss.py commit `8259b16a5d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 26 02:49:02 2024 +1100 gemma commit `df8034d185` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:48:34 2024 +1100 Update llama.py commit `db2b387b22` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:47:45 2024 +1100 llama commit `36d29078cd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:35:39 2024 +1100 Update gemma.py commit `4cc14b2074` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:30:59 2024 +1100 Update gemma.py commit `bc428f4e7b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:29:01 2024 +1100 Update gemma.py commit `7c958ab9e7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:25:57 2024 +1100 Update gemma.py commit `d530f95135` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:25:15 2024 +1100 Update gemma.py commit `6ad44835d2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:22:35 2024 +1100 Update gemma.py commit `46723124f7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:21:03 2024 +1100 Update gemma.py commit `ad1ce483db` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:20:54 2024 +1100 Update gemma.py commit `47d4a33f41` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:20:09 2024 +1100 Update gemma.py commit `6b531eff93` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:18:23 2024 +1100 Update gemma.py commit `147d129772` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:17:16 2024 +1100 Update gemma.py commit `884fadc744` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:14:57 2024 +1100 Update gemma.py commit `9d648cbccf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:14:38 2024 +1100 Update gemma.py commit `6238f16625` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:08:04 2024 +1100 Update gemma.py commit `9a20dff5d2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 04:07:47 2024 +1100 Update gemma.py commit `94833602da` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:55:44 2024 +1100 Update gemma.py commit `40c244a7bc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:53:40 2024 +1100 Update gemma.py commit `0e1578dedb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:51:32 2024 +1100 Update gemma.py commit `33a72ba122` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:50:59 2024 +1100 Update gemma.py commit `890d73e6d0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:49:55 2024 +1100 Update gemma.py commit `33eeb7add2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:44:22 2024 +1100 Update gemma.py commit `096a4192fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:39:40 2024 +1100 Update gemma.py commit `f270f377ab` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:39:31 2024 +1100 Update gemma.py commit `765e54fe55` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:38:18 2024 +1100 Update gemma.py commit `29721a86d1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:37:30 2024 +1100 Update gemma.py commit `208e2c1189` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:32:23 2024 +1100 Update gemma.py commit `aff1db7a4c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:22:24 2024 +1100 Update gemma.py commit `98903e72c2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:21:37 2024 +1100 Update gemma.py commit `20da5547f5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:19:34 2024 +1100 Update gemma.py commit `20e9ca2fac` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:16:55 2024 +1100 Update gemma.py commit `842b310767` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:16:34 2024 +1100 Update gemma.py commit `11067cf849` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:14:02 2024 +1100 Update gemma.py commit `6725dc93f1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:13:18 2024 +1100 Update gemma.py commit `baf11e0e34` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:12:07 2024 +1100 Update gemma.py commit `e0ada344ac` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:11:20 2024 +1100 Update gemma.py commit `39a58f7d2e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:07:20 2024 +1100 Update gemma.py commit `c96e8c28f8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:03:31 2024 +1100 Update gemma.py commit `10d03d8315` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:03:23 2024 +1100 Update gemma.py commit `45a33bae1c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:01:56 2024 +1100 Update gemma.py commit `77fea80a1b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:01:30 2024 +1100 Update gemma.py commit `e3517f0248` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 03:00:22 2024 +1100 Update gemma.py commit `03c7f211a6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:49:14 2024 +1100 Update gemma.py commit `105a0325f1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:42:56 2024 +1100 Update gemma.py commit `9d9e38a74a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:41:24 2024 +1100 Update gemma.py commit `49a124243e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:37:48 2024 +1100 Update gemma.py commit `5c31fd1e5b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:34:18 2024 +1100 Update gemma.py commit `c0f6b6c0cc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:31:22 2024 +1100 Update gemma.py commit `911745377a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:30:06 2024 +1100 Update gemma.py commit `f73698859e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:28:01 2024 +1100 Update gemma.py commit `e56eb3f406` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:26:57 2024 +1100 Update gemma.py commit `045bce2fa5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:25:39 2024 +1100 Update gemma.py commit `a847719499` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:18:43 2024 +1100 Update gemma.py commit `c058d1aa96` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:17:45 2024 +1100 Update gemma.py commit `dfb2e250f6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:17:12 2024 +1100 Update gemma.py commit `f2fca2d6c1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:10:35 2024 +1100 Update gemma.py commit `ae473e0f2b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:09:08 2024 +1100 rope commit `ce2d74732d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:07:29 2024 +1100 Update llama.py commit `ed5042aa07` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:04:11 2024 +1100 Update llama.py commit `42840647d0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:02:59 2024 +1100 Update llama.py commit `ecd21b375b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 02:02:02 2024 +1100 Update llama.py commit `5bf8cacebd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:58:33 2024 +1100 Update gemma.py commit `66a4380d08` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:50:42 2024 +1100 Update gemma.py commit `b85b45e7a0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:49:39 2024 +1100 Update gemma.py commit `72870a1ed3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:48:02 2024 +1100 Update gemma.py commit `7b903368ab` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:45:44 2024 +1100 Update gemma.py commit `0c2d7e503b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:39:25 2024 +1100 Update cross_entropy_loss.py commit `9ae5abc83f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:12:44 2024 +1100 Update gemma.py commit `3907ea9003` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:10:12 2024 +1100 Update gemma.py commit `c4c8558a90` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:04:09 2024 +1100 Update gemma.py commit `e0e96ef5ce` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 01:03:04 2024 +1100 Update gemma.py commit `df22d0cb3a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 00:46:58 2024 +1100 Update gemma.py commit `b9d9aa1b07` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 25 00:44:44 2024 +1100 Update gemma.py commit `785760ba0d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:43:00 2024 +1100 Update gemma.py commit `103b1cdbed` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:42:09 2024 +1100 Update gemma.py commit `b482ce13e9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:40:46 2024 +1100 Update gemma.py commit `7be57334cb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:39:28 2024 +1100 Update gemma.py commit `2242bdf0a3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:30:14 2024 +1100 Update gemma.py commit `a066c348f2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 22:24:43 2024 +1100 Update llama.py commit `b3d7e61608` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:20:37 2024 +1100 Update gemma.py commit `5b5652de36` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:17:24 2024 +1100 Update gemma.py commit `0f5cc839f3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:15:25 2024 +1100 Update gemma.py commit `6011a490c6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:13:24 2024 +1100 Update gemma.py commit `6d94cf88a7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:11:23 2024 +1100 Update gemma.py commit `c706447fd4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 20:04:31 2024 +1100 Update gemma.py commit `ed3f139a9c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:55:22 2024 +1100 Update gemma.py commit `b7ba95857f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:47:10 2024 +1100 Update gemma.py commit `f0b21f9b7b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:45:20 2024 +1100 Update gemma.py commit `762790d6a9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:35:03 2024 +1100 Update gemma.py commit `b174e54507` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:33:04 2024 +1100 Update gemma.py commit `47c6feaf89` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:12:52 2024 +1100 Update gemma.py commit `b2b658cbee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 19:10:36 2024 +1100 Update gemma.py commit `407205dd87` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 18:58:31 2024 +1100 Update gemma.py commit `82e45c02a5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 18:25:25 2024 +1100 revert commit `94201755db` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 18:17:38 2024 +1100 revert commit `d5e625b674` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 18:04:40 2024 +1100 Update cross_entropy_loss.py commit `ba19344fb9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 18:03:45 2024 +1100 Update cross_entropy_loss.py commit `7916872408` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:48:19 2024 +1100 Update llama.py commit `49129bc66c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:43:19 2024 +1100 Update gemma.py commit `ebbd4756d4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:41:53 2024 +1100 Update gemma.py commit `68e280405c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:40:21 2024 +1100 Update gemma.py commit `9af090f6ea` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:32:04 2024 +1100 Update gemma.py commit `61add47f37` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:31:10 2024 +1100 Update gemma.py commit `de96285442` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:29:51 2024 +1100 Update gemma.py commit `0abbcdc15a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:29:09 2024 +1100 Update gemma.py commit `17036a6e54` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:27:49 2024 +1100 Update llama.py commit `8b7de591c1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:26:19 2024 +1100 Update gemma.py commit `041aa7d909` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:18:09 2024 +1100 Update gemma.py commit `71483e6c8d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:17:26 2024 +1100 Update gemma.py commit `8cb12078f9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:15:44 2024 +1100 Update gemma.py commit `23e6ebf14d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:13:40 2024 +1100 Update gemma.py commit `a6752f3f16` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:12:00 2024 +1100 Update gemma.py commit `9c565580fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:09:03 2024 +1100 Update gemma.py commit `cd479b4374` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:05:59 2024 +1100 Update cross_entropy_loss.py commit `0080357a88` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:03:54 2024 +1100 Update gemma.py commit `e8de606be9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:02:54 2024 +1100 Update gemma.py commit `35adcbf83c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 17:02:14 2024 +1100 Update gemma.py commit `7e33aa2520` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 16:59:24 2024 +1100 Update gemma.py commit `4c7d21e41b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 16:57:16 2024 +1100 Update gemma.py commit `3feae56451` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 16:55:21 2024 +1100 Update gemma.py commit `c060bad7ce` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 16:47:39 2024 +1100 Update gemma.py commit `8ffaf5f109` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 15:44:10 2024 +1100 Update gemma.py commit `be098122f1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 15:42:12 2024 +1100 Update llama.py commit `8a618c60ed` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:29:15 2024 +1100 pos commit `ef235c3508` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:27:39 2024 +1100 Update gemma.py commit `a48adb0a83` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:27:31 2024 +1100 Update gemma.py commit `8920cda9a7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:13:31 2024 +1100 position_ids commit `a1eab801fa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:06:48 2024 +1100 Update gemma.py commit `c07aff5188` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:05:18 2024 +1100 Update gemma.py commit `4c6e122caa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 14:02:26 2024 +1100 norm commit `5c5bb53241` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 13:59:30 2024 +1100 Update llama.py commit `0e1826dd8c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 13:55:37 2024 +1100 Update llama.py commit `893b5dfe2b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 13:54:00 2024 +1100 revert commit `9ab87f3648` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 13:33:05 2024 +1100 Update cross_entropy_loss.py commit `d2dc658077` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 13:22:17 2024 +1100 Update geglu.py commit `1fcbd61f76` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:40:56 2024 +1100 Update cross_entropy_loss.py commit `fadcb311c3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:40:43 2024 +1100 Update llama.py commit `1a2a10d028` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:38:25 2024 +1100 Update llama.py commit `17a1a855e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:33:46 2024 +1100 CE commit `097629108f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:30:43 2024 +1100 Update llama.py commit `30cc4ffd67` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:27:21 2024 +1100 Update llama.py commit `20227ba5c7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:14:29 2024 +1100 Update llama.py commit `73a8616f99` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 03:03:00 2024 +1100 Update llama.py commit `bd4fd22f34` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:58:05 2024 +1100 Update cross_entropy_loss.py commit `2814f02087` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:33:28 2024 +1100 Update cross_entropy_loss.py commit `dd321363f2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:30:49 2024 +1100 Update cross_entropy_loss.py commit `574b2f787f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:29:33 2024 +1100 Update cross_entropy_loss.py commit `6184f770e7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:10:38 2024 +1100 Update cross_entropy_loss.py commit `f5f3d6794e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:08:26 2024 +1100 Update cross_entropy_loss.py commit `88bf684c61` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:06:11 2024 +1100 Update cross_entropy_loss.py commit `603c71c7f0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 24 02:02:11 2024 +1100 Fast CE Loss commit `25339b71f7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 18:12:15 2024 +1100 Update fast_lora.py commit `ebfc8f8a55` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 17:57:42 2024 +1100 Update fast_lora.py commit `6ab5eb75f4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 17:55:58 2024 +1100 Update llama.py commit `f999bcd52d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 17:46:33 2024 +1100 Update llama.py commit `13d0cee4f6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 17:37:17 2024 +1100 Update llama.py commit `b66f6dbfd3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 17:28:29 2024 +1100 Update llama.py commit `5866b08f71` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 04:11:08 2024 +1100 gemma commit `dd478be2fa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 03:59:09 2024 +1100 Update llama.py commit `2ff33fb270` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 03:52:24 2024 +1100 Update llama.py commit `4c9f366688` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 03:27:42 2024 +1100 Update cross_entropy_loss.py commit `cfebc8d979` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 03:27:03 2024 +1100 Update llama.py commit `4b009aad33` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 03:16:50 2024 +1100 Update llama.py commit `6f340f5417` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:38:16 2024 +1100 Update fast_lora.py commit `ff27a824fa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:33:37 2024 +1100 Update llama.py commit `5d728270d3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:28:05 2024 +1100 Update llama.py commit `e743e58cb6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:23:25 2024 +1100 Update gemma.py commit `879bdd2efb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:20:26 2024 +1100 Update gemma.py commit `9052a85f92` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:19:06 2024 +1100 Update gemma.py commit `bad295a413` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:15:12 2024 +1100 Update llama.py commit `b30d2502cc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:12:02 2024 +1100 Update llama.py commit `11233cb3d4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:10:11 2024 +1100 model_type commit `76de9c1fe3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:05:31 2024 +1100 FastGemmaModel commit `c0d32de795` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 02:02:39 2024 +1100 Update fast_lora.py commit `bd2fa264c9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 01:57:34 2024 +1100 Update mapper.py commit `9a1b28d691` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 01:55:25 2024 +1100 Update pyproject.toml commit `0beaf18908` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 23 01:50:51 2024 +1100 Gemma commit `0659d90d95` Merge: `7372768` `1b7bf71` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 17:49:57 2024 +1100 Merge branch 'main' into nightly commit `7372768d14` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 03:29:49 2024 +1100 original commit `10d9d56434` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 03:25:46 2024 +1100 spaces commit `edd03f66fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 03:14:27 2024 +1100 trainer commit `028ee5ca06` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 02:05:33 2024 +1100 save commit `917f791ab7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 00:53:17 2024 +1100 Update save.py commit `bf3e10b26e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 21 00:48:18 2024 +1100 Update save.py commit `d266332141` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 23:28:37 2024 +1100 Update save.py commit `4d1e575047` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 20:01:33 2024 +1100 Update __init__.py commit `6aac6c4be8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 19:50:29 2024 +1100 Update save.py commit `83d906a2c0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 18:35:11 2024 +1100 Update save.py commit `5d88ffefd5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 18:00:51 2024 +1100 Update save.py commit `4c9be6d057` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 17:12:18 2024 +1100 Update save.py commit `d3eac595a1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 17:11:33 2024 +1100 Update save.py commit `632705b9b0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 17:00:19 2024 +1100 Update save.py commit `8f60bf57be` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 15:59:38 2024 +1100 Update save.py commit `164fa807fe` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:44:17 2024 +1100 Update save.py commit `11e04a7057` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:42:47 2024 +1100 Update save.py commit `d2e76580ae` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:41:40 2024 +1100 Update save.py commit `3e3dc37640` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:41:34 2024 +1100 Update save.py commit `d9751e6553` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:35:07 2024 +1100 Update save.py commit `44659a05a3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 04:13:58 2024 +1100 Update save.py commit `34998914ac` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 03:35:19 2024 +1100 saving commit `3728b8e876` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 02:19:36 2024 +1100 Update save.py commit `c20bb30710` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 02:07:01 2024 +1100 Update save.py commit `2c207b4989` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 20 00:10:04 2024 +1100 llama.cpp bugs commit `0ffd7b46f3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 19:39:47 2024 +1100 linking commit `ad0c2bcd87` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 13:08:03 2024 +1100 Update save.py commit `0c1b71ac63` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 12:51:25 2024 +1100 Update save.py commit `14db89f4fc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 04:13:40 2024 +1100 Update save.py commit `e31071f368` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 04:01:27 2024 +1100 Update save.py commit `aa45208f5a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 03:39:09 2024 +1100 Update save.py commit `28de27dd8d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 03:22:21 2024 +1100 PeftModel token + saving commit `99cdf0bd5a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 02:45:54 2024 +1100 Update save.py commit `71555c7678` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 19 02:37:43 2024 +1100 Update save.py commit `89d2418cd7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 23:35:18 2024 +1100 Update save.py commit `3e89d4e7ec` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 23:31:24 2024 +1100 Update save.py commit `457c044473` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 23:25:36 2024 +1100 install commit `22378e95cc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 23:12:37 2024 +1100 Update pyproject.toml commit `3d789cb09e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 22:59:37 2024 +1100 Update save.py commit `31a62c52d3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 22:54:38 2024 +1100 trainer commit `870edf3534` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 22:52:51 2024 +1100 Update save.py commit `b6a6e90b7a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 22:03:22 2024 +1100 Update save.py commit `02b7b7f3f2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 20:28:56 2024 +1100 Update save.py commit `53f6a07b92` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 20:22:52 2024 +1100 Update save.py commit `7ee8243e8b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 20:19:28 2024 +1100 Update save.py commit `94b5c58c2e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 20:11:51 2024 +1100 Update save.py commit `c6ad5f97a3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 20:09:11 2024 +1100 Update loader.py commit `cc2764cddd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:54:34 2024 +1100 Update save.py commit `0af3a1b143` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:44:59 2024 +1100 Update save.py commit `ef8abf4c6e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:25:56 2024 +1100 apache commit `164b950cad` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:24:35 2024 +1100 spaces commit `7aedc92637` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:21:02 2024 +1100 slashes commit `7573ee2c22` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:15:41 2024 +1100 slash commit `e51a381305` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 19:08:04 2024 +1100 globals commit `c660879cdc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 18:30:03 2024 +1100 spaces commit `dcbbab3c22` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 18:25:19 2024 +1100 spaces commit `5edae1cd79` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 18:21:54 2024 +1100 readme commit `22f1f52513` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 18:18:12 2024 +1100 Update llama.py commit `2e65e63678` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 18:09:38 2024 +1100 saving bugs commit `c9a524a99a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 18 04:15:20 2024 +1100 Bugs commit `6d3a3b4286` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 15 19:24:07 2024 +1100 Fix RoPE precision issues commit `84b37f8548` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 15 03:53:58 2024 +1100 Update mapper.py commit `bd4a701e7a` Merge: `629c39d` `0439b85` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 15 03:35:47 2024 +1100 Merge branch 'main' into nightly commit `629c39d2e5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 23:17:38 2024 +1100 Update mistral.py commit `227c26ca44` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 23:15:45 2024 +1100 Update llama.py commit `b660132380` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 23:11:06 2024 +1100 Saving, LlamaRotaryEmbedding issues commit `91a3c43468` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 17:58:41 2024 +1100 Update chat_templates.py commit `efbb1e6049` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 17:56:14 2024 +1100 patch tokenizer commit `5f5910ffee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 17:45:02 2024 +1100 Update chat_templates.py commit `d40a12852e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 17:31:33 2024 +1100 Update chat_templates.py commit `7c713cb58c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 17:27:03 2024 +1100 Update chat_templates.py commit `b28a383d1c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 04:20:41 2024 +1100 Update chat_templates.py commit `4f20e20e28` Merge: `2cdf43d` `474fd32` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 14 03:25:40 2024 +1100 Merge branch 'main' into nightly commit `2cdf43d8b7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 12 04:28:41 2024 +1100 Chat Templates commit `acd635aa0d` Merge: `b7c5296` `3d5cf37` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 11 16:42:37 2024 +1100 Merge branch 'main' into nightly commit `b7c52963ad` Merge: `868fb27` `99b8d23` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 9 03:52:36 2024 +1100 Merge branch 'main' into nightly commit `868fb27e11` Merge: `601dc9e` `b7f24e8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 16:50:19 2024 +1100 Merge branch 'main' into nightly commit `601dc9ec4b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 03:39:45 2024 +1100 Update llama.py commit `31de486f1c` Merge: `81128a4` `25cfc7f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 03:39:19 2024 +1100 revert commit `81128a4504` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 03:00:53 2024 +1100 Update llama.py commit `998097394a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 02:57:09 2024 +1100 Update llama.py commit `277ca9eecf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 02:52:43 2024 +1100 Update llama.py commit `d6ab9c92d7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 02:49:26 2024 +1100 Update llama.py commit `e094914b0e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 8 02:30:49 2024 +1100 Update llama.py commit `9a54a6f05e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 20:15:19 2024 +1100 Update llama.py commit `e94647c2dc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 19:25:46 2024 +1100 Update llama.py commit `e8593b7103` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 18:25:08 2024 +1100 Update llama.py commit `1065936ecb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 18:21:36 2024 +1100 Update llama.py commit `60b47f6130` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 18:18:43 2024 +1100 Update llama.py commit `e43e819a57` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 18:02:58 2024 +1100 Update llama.py commit `1e77ab25b6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 17:43:51 2024 +1100 Update llama.py commit `b36e0bfc89` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 17:34:59 2024 +1100 Update llama.py commit `ab0a3c9976` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 04:40:19 2024 +1100 Update mistral.py commit `2b346dc498` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 03:45:07 2024 +1100 Update save.py commit `17a6b12ee3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 03:41:32 2024 +1100 Update save.py commit `7da7afcc78` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 03:20:55 2024 +1100 __version__ commit `9e2b00e167` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 03:19:33 2024 +1100 __version__ commit `9c3849fc1d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 03:18:02 2024 +1100 Update pyproject.toml commit `bfb3ea7179` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 02:57:54 2024 +1100 Update save.py commit `213cfee903` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 02:57:24 2024 +1100 Update save.py commit `8b52dc027e` Merge: `8e9d9c3` `bb66faa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 02:50:35 2024 +1100 Merge branch 'main' into nightly commit `8e9d9c38b1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Feb 7 01:48:31 2024 +1100 SWA inference commit `d0b1144423` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 6 18:53:32 2024 +1100 Fix llm_int8_skip_modules commit `39a2a7c57d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 6 18:50:48 2024 +1100 Fix SWA inference commit `aa0427f898` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 6 02:11:06 2024 +1100 Update save.py commit `262289d5a5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 6 01:54:22 2024 +1100 Update save.py commit `53ca91f744` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Feb 6 01:30:48 2024 +1100 Update save.py commit `e487abd94a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 19:07:00 2024 +1100 Update save.py commit `797a87a3e3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 18:53:27 2024 +1100 Update save.py commit `e9031ceabe` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 18:48:40 2024 +1100 mistral swa commit `33192d6dc8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 18:05:25 2024 +1100 Update save.py commit `03ed3a83d5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 17:38:01 2024 +1100 Torch 2.2.0 commit `a50daa19d8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 17:32:43 2024 +1100 Update save.py commit `d69cef9f23` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 17:16:20 2024 +1100 Update save.py commit `1750b13a63` Merge: `63ed23a` `efa0d23` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Feb 5 02:29:56 2024 +1100 Merge branch 'main' into nightly commit `63ed23ae98` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 14:02:26 2024 +1100 Update utils.py commit `990068b977` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 13:50:21 2024 +1100 Update utils.py commit `6ab4019e4a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 04:07:47 2024 +1100 Update llama.py commit `7ab3426608` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:57:29 2024 +1100 Update llama.py commit `201d90c3ba` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:54:12 2024 +1100 Update llama.py commit `00242d5f89` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:52:08 2024 +1100 Update llama.py commit `3bfb0ebf9d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:50:31 2024 +1100 Update llama.py commit `71899d7060` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:48:18 2024 +1100 Update llama.py commit `9cd2517949` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:46:30 2024 +1100 Update llama.py commit `24431b4347` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:45:25 2024 +1100 Update llama.py commit `63816fc119` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:42:41 2024 +1100 Update llama.py commit `7166b11599` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:41:15 2024 +1100 Update llama.py commit `d867b9bbdf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:27:15 2024 +1100 Update llama.py commit `9a5ebef148` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:23:26 2024 +1100 Update llama.py commit `65270cec60` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:13:42 2024 +1100 Update llama.py commit `80fa8e93c9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:12:07 2024 +1100 Update llama.py commit `36b400e598` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:08:49 2024 +1100 Update llama.py commit `4a5d3b1de7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 03:05:49 2024 +1100 Update llama.py commit `88a695da3a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:49:51 2024 +1100 Update llama.py commit `ac9bc79251` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:42:30 2024 +1100 Update llama.py commit `8c8685eeef` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:40:30 2024 +1100 Update llama.py commit `607dfa1d0e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:35:17 2024 +1100 Update llama.py commit `54802ecbb9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:32:21 2024 +1100 New version commit `711e5c0922` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:19:42 2024 +1100 attention_mask commit `74d7fc65c6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:03:23 2024 +1100 SDPA commit `fcb884643b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 02:02:51 2024 +1100 Update llama.py commit `aa032fc80d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 01:46:04 2024 +1100 Update mistral.py commit `31578f2010` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Feb 4 01:42:46 2024 +1100 fast inference commit `257cd7d531` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 23:22:21 2024 +1100 Update llama.py commit `e500b785f7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 23:12:16 2024 +1100 Update llama.py commit `cf0fae9a55` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 23:01:56 2024 +1100 Update llama.py commit `381b991c45` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 22:42:11 2024 +1100 Update llama.py commit `665908eb70` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 22:28:20 2024 +1100 Update llama.py commit `5534f8a8f1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 20:22:42 2024 +1100 more temp matrices commit `68db1c7af7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 19:44:36 2024 +1100 fast inference again commit `d76f583349` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 19:30:22 2024 +1100 Update mistral.py commit `9225dd6708` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 19:19:13 2024 +1100 Update llama.py commit `8270821269` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 19:08:42 2024 +1100 Update llama.py commit `1c6e1f18b4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 18:32:18 2024 +1100 Update llama.py commit `dd03abedd6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 18:20:06 2024 +1100 Update llama.py commit `522f6dbfd2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 18:07:55 2024 +1100 Update llama.py commit `ad357dec89` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 03:52:25 2024 +1100 fast inference + saving config.json commit `a78d6fba7e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 03:34:02 2024 +1100 Update llama.py commit `fa3d23406e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 03:10:39 2024 +1100 Update utils.py commit `8e3f0296be` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 02:55:22 2024 +1100 Update utils.py commit `404e177adb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 02:30:47 2024 +1100 Update utils.py commit `a81d193830` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 02:20:00 2024 +1100 Update utils.py commit `4436a1099d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:50:48 2024 +1100 Update llama.py commit `a15ffc95e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:48:03 2024 +1100 Update llama.py commit `0be46406d8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:39:39 2024 +1100 Update llama.py commit `e911047188` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:35:45 2024 +1100 Update llama.py commit `c934c16e21` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:29:50 2024 +1100 Update llama.py commit `705bbba5c5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Feb 3 01:22:53 2024 +1100 Update llama.py commit `9c2bed35b9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 23:45:02 2024 +1100 Update llama.py commit `ea9b4eea0c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 23:40:52 2024 +1100 Update llama.py commit `c6ad936f88` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 23:32:58 2024 +1100 Update llama.py commit `03df291110` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 23:32:36 2024 +1100 Update llama.py commit `dc2740416b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 22:26:47 2024 +1100 Update llama.py commit `b33c92d1bd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 22:22:42 2024 +1100 Update llama.py commit `923c6bad14` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 22:20:45 2024 +1100 Update llama.py commit `b3703926ca` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 22:17:02 2024 +1100 Update llama.py commit `1b274212e1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:35:00 2024 +1100 Update llama.py commit `9df19aeb9f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:32:21 2024 +1100 past_key_values commit `82eea75cde` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:29:35 2024 +1100 torch compile commit `e748078095` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:24:44 2024 +1100 Update llama.py commit `9146fa443d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:24:28 2024 +1100 Update llama.py commit `a5a123db08` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:20:14 2024 +1100 Update llama.py commit `82ead808d5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:19:15 2024 +1100 Update llama.py commit `1497521e5c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:06:52 2024 +1100 Update mistral.py commit `c3c454a729` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 20:06:31 2024 +1100 Update llama.py commit `f299c9cde1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 19:44:52 2024 +1100 Update llama.py commit `0e1d67d4e1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 19:07:41 2024 +1100 fast inference commit `168ded977e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 18:28:05 2024 +1100 Update llama.py commit `bf441055ca` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 18:14:31 2024 +1100 faster inference commit `7c2b04254c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 16:59:02 2024 +1100 Update llama.py commit `ca99d7c194` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 16:46:30 2024 +1100 Update mistral.py commit `40e8848c4e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 15:56:56 2024 +1100 Update llama.py commit `6cc9835021` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 13:52:12 2024 +1100 Update llama.py commit `7ad1a1fa62` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 04:18:34 2024 +1100 Update llama.py commit `0b661a23b9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Feb 2 03:55:36 2024 +1100 Update llama.py commit `da64d3403c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:54:01 2024 +1100 Update llama.py commit `abc47836dc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:41:32 2024 +1100 Update llama.py commit `f231c4f395` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:41:23 2024 +1100 Update llama.py commit `5f3c51b394` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:41:11 2024 +1100 Update llama.py commit `e0ea238256` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:40:36 2024 +1100 Update llama.py commit `73f63d6884` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:38:07 2024 +1100 Update llama.py commit `e4b5e38800` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 23:17:53 2024 +1100 inference commit `334c5ed1f0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 22:48:54 2024 +1100 Update llama.py commit `cd39f6108f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 22:34:33 2024 +1100 lm_head commit `24c4c37b7c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 22:21:35 2024 +1100 revert commit `e791db9b06` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 20:12:59 2024 +1100 Update llama.py commit `1793a16c8f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 20:03:54 2024 +1100 faster inference commit `19fb50e244` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:56:54 2024 +1100 Update utils.py commit `b83cea7cb4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:56:06 2024 +1100 Update llama.py commit `8920cafbe7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:46:38 2024 +1100 inference commit `38b59825b1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:36:36 2024 +1100 Update llama.py commit `e8ec80a4c2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:19:44 2024 +1100 Update llama.py commit `a2cb7a113b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 19:07:30 2024 +1100 Update llama.py commit `a5ee70b63f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 18:34:43 2024 +1100 Update llama.py commit `e2f0dd8683` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 18:16:46 2024 +1100 Update llama.py commit `3f0ddf0c7c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 18:16:15 2024 +1100 Update llama.py commit `e90b3bf192` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 17:59:21 2024 +1100 Update llama.py commit `57044509ad` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 17:44:22 2024 +1100 Update llama.py commit `cf4b58eeb6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 17:20:02 2024 +1100 inference commit `648c79ec1b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 03:47:51 2024 +1100 faster inference commit `20f19391e6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 03:05:02 2024 +1100 Update mistral.py commit `e2f72fe52f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 03:04:46 2024 +1100 Revert commit `329f80ac4c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 02:47:57 2024 +1100 Update llama.py commit `713a95ca0e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Feb 1 02:46:26 2024 +1100 Inference commit `acbdef7ff5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 20:16:08 2024 +1100 padding commit `0dc26ed98a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 20:06:28 2024 +1100 Update llama.py commit `9f31254539` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 20:02:21 2024 +1100 Fix SDPA commit `7227de48cf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 19:48:33 2024 +1100 Update llama.py commit `5edad55b5d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 19:44:19 2024 +1100 Update llama.py commit `c928c57aa9` Merge: `5da0555` `2f55935` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 19:38:27 2024 +1100 Merge branch 'main' into nightly commit `5da05558a0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:50:47 2024 +1100 past_key_value commit `d347db0944` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:46:11 2024 +1100 Update loader.py commit `c0edaa46db` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:40:55 2024 +1100 revert inference commit `55fe6052ca` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:36:40 2024 +1100 if past_key_value is not None and q_len == 1: commit `248887b51c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:35:24 2024 +1100 LlamaAttention_fast_forward_inference commit `c68a3bc9ec` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 03:28:30 2024 +1100 Update loader.py commit `44168377f3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 31 01:32:56 2024 +1100 Update rope_embedding.py commit `270df81b60` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 17:33:08 2024 +1100 Remove fast path commit `b8f665bf22` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 17:26:45 2024 +1100 fast lm_head commit `ed5a653ecf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 04:16:39 2024 +1100 Update mistral.py commit `e0bad0eec5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 04:10:14 2024 +1100 Fix inference commit `71725aeea5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 03:23:42 2024 +1100 Update __init__.py commit `3ddda6f492` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 30 03:08:46 2024 +1100 Update mistral.py commit `6f74c98fbc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 23:19:43 2024 +1100 Update utils.py commit `a7bfeec919` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 23:14:51 2024 +1100 Update utils.py commit `7c87d60bc1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 23:07:40 2024 +1100 Update utils.py commit `bb364204cc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 20:09:59 2024 +1100 Update llama.py commit `4700d51916` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 19:55:40 2024 +1100 Fast inference repatch commit `58cabcbc06` Merge: `01b8162` `a3a2ad9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 17:59:34 2024 +1100 Merge branch 'main' into nightly commit `01b8162244` Merge: `5cfea20` `90309ca` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 17:49:57 2024 +1100 Merge branch 'main' into nightly commit `5cfea20129` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 17:35:19 2024 +1100 Update llama.py commit `25a88ea003` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 16:52:04 2024 +1100 Update llama.py commit `03ca52dc94` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 03:43:39 2024 +1100 saving commit `01e5c305f9` Merge: `e10e488` `a16bc73` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 03:43:19 2024 +1100 Merge branch 'main' into nightly commit `e10e48893b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 03:43:17 2024 +1100 Update save.py commit `5bd916b91b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 03:43:05 2024 +1100 Update mistral.py commit `11ba2c520b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 03:34:27 2024 +1100 Mistral patch commit `498dfb8acc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 02:42:08 2024 +1100 print commit `a3d2b9b778` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 02:15:52 2024 +1100 Update save.py commit `9e00cc287a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:59:29 2024 +1100 Update save.py commit `460de24ea2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:57:11 2024 +1100 Update save.py commit `b060d7b621` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:56:44 2024 +1100 Update save.py commit `74b69775c2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:51:57 2024 +1100 Update save.py commit `fef0589d6e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:51:45 2024 +1100 Update save.py commit `9dec4b3f08` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 29 00:49:31 2024 +1100 Update save.py commit `ee0bf6fe66` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 20:07:17 2024 +1100 Update save.py commit `c69c166b82` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 18:12:11 2024 +1100 patch_saving_functions commit `ac02ba6c38` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 18:08:52 2024 +1100 Update save.py commit `20e524a596` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 18:04:35 2024 +1100 Update save.py commit `788e695180` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 17:48:34 2024 +1100 Patch saving commit `31222ced74` Merge: `893aab0` `af33224` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 17:48:28 2024 +1100 Merge branch 'main' into nightly commit `893aab0e57` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 17:20:49 2024 +1100 Update dpo.py commit `d5c852e711` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 16:47:33 2024 +1100 Update llama.py commit `ddb48efd33` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 16:43:10 2024 +1100 Update llama.py commit `5fe166d32e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 16:40:59 2024 +1100 Update llama.py commit `2d64e0a904` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 16:37:21 2024 +1100 Update mistral.py commit `2f73cb4049` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 16:31:01 2024 +1100 Update llama.py commit `6aa46ffff6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 15:13:54 2024 +1100 Update llama.py commit `ee6f5096ed` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 13:53:06 2024 +1100 attention mask commit `f1b0fd0848` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 13:37:38 2024 +1100 Update mistral.py commit `7663a32753` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:29:52 2024 +1100 Update save.py commit `a158003625` Merge: `36829b7` `e2bbd38` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:29:35 2024 +1100 Merge branch 'nightly' of https://github.com/unslothai/unsloth into nightly commit `36829b7ec4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:27:10 2024 +1100 Update save.py commit `917ce15861` Merge: `166f8c8` `a81aff2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:20:15 2024 +1100 Merge branch 'main' into nightly commit `166f8c812e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:18:19 2024 +1100 attention mask commit `6c7f0dbcb4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:08:19 2024 +1100 Update llama.py commit `c836ed7f3d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 04:08:03 2024 +1100 Update mistral.py commit `12d57e5308` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 03:59:23 2024 +1100 labels commit `4c5ebcc960` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 03:56:54 2024 +1100 Update llama.py commit `6a027a8292` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 03:55:23 2024 +1100 Update llama.py commit `9f9739cbac` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 03:49:54 2024 +1100 attention_mask commit `2bd77e7277` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 02:34:03 2024 +1100 Update fast_lora.py commit `a1e5aca7cc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 01:27:46 2024 +1100 Update fast_lora.py commit `a2f705d65b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 01:17:27 2024 +1100 Update fast_lora.py commit `d01ba458df` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 28 00:45:55 2024 +1100 Update fast_lora.py commit `6fa0635971` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 23:20:41 2024 +1100 Update fast_lora.py commit `e094af81a1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 20:39:19 2024 +1100 Update fast_lora.py commit `c74e1af85c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 20:37:43 2024 +1100 Update fast_lora.py commit `363ffba1c2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 20:17:08 2024 +1100 Update fast_lora.py commit `510c85f412` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 20:04:55 2024 +1100 Update swiglu.py commit `35daafdd6e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:53:57 2024 +1100 Update fast_lora.py commit `6201f7681f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:38:28 2024 +1100 Update fast_lora.py commit `86a1c9788b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:29:53 2024 +1100 Update fast_lora.py commit `8e0e4ccdc6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:18:59 2024 +1100 Update fast_lora.py commit `2e4c59deaf` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:05:02 2024 +1100 Update swiglu.py commit `85e87d9ba2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 19:03:04 2024 +1100 Swiglu commit `83b6937285` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 18:21:29 2024 +1100 Update fast_lora.py commit `3d3e7f5b42` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 18:15:34 2024 +1100 Update fast_lora.py commit `d3f3b6fc49` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 18:00:41 2024 +1100 Update fast_lora.py commit `e77d7c069f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 18:00:22 2024 +1100 Update fast_lora.py commit `f7d11d10f8` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 17:47:54 2024 +1100 Update fast_lora.py commit `8ed03f5f45` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 17:38:30 2024 +1100 Update pyproject.toml commit `af65cb0d3d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:50:47 2024 +1100 Works? commit `fb5333726a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:49:36 2024 +1100 Update llama.py commit `a59ec7903c` Merge: `704e36a` `7da0c50` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:48:40 2024 +1100 Merge branch 'main' into nightly commit `704e36a64e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:48:34 2024 +1100 Revert "Update llama.py" This reverts commit `a208ec46e0`. commit `a208ec46e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:48:03 2024 +1100 Update llama.py commit `bd2ff90817` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:47:42 2024 +1100 Update llama.py commit `a3d892a15b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:47:01 2024 +1100 Update llama.py commit `b89599a3f7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:46:47 2024 +1100 Update llama.py commit `baeea64917` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:20:58 2024 +1100 Update save.py commit `ecdbb28dcb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:19:59 2024 +1100 Update save.py commit `393341f211` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:18:25 2024 +1100 Update swiglu.py commit `47babc780a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:16:38 2024 +1100 Update fast_lora.py commit `9edc309f59` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:15:37 2024 +1100 Update llama.py commit `a27ac6175e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 04:03:47 2024 +1100 Update utils.py commit `7fb64e0e31` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:58:21 2024 +1100 Update fast_lora.py commit `e3bd0bb74f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:51:53 2024 +1100 Update save.py commit `ba847f5411` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:38:33 2024 +1100 Update fast_lora.py commit `c57495df6f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:38:19 2024 +1100 Update fast_lora.py commit `d379bb8cf5` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:36:27 2024 +1100 Update fast_lora.py commit `421ed33b42` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:23:50 2024 +1100 Update fast_lora.py commit `83ceb11cc9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 03:12:53 2024 +1100 Update fast_lora.py commit `f3da0d2e4a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 02:52:54 2024 +1100 Update fast_lora.py commit `4ae3ad3bce` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 02:34:21 2024 +1100 Update fast_lora.py commit `c74d3dd3b9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 02:22:15 2024 +1100 Update fast_lora.py commit `0a1aa98d3c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 02:19:00 2024 +1100 Update fast_lora.py commit `f7760719f0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 01:28:03 2024 +1100 Update fast_lora.py commit `38bff800b6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 27 01:16:31 2024 +1100 Update fast_lora.py commit `485d54fd70` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 23:48:17 2024 +1100 Update fast_lora.py commit `ce08a15cee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 23:33:17 2024 +1100 Update fast_lora.py commit `b4492eb29c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 23:14:44 2024 +1100 Update fast_lora.py commit `3865c8cd68` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 23:06:06 2024 +1100 Update fast_lora.py commit `e9e34f1d4d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 22:29:38 2024 +1100 Update fast_lora.py commit `d96f665510` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 22:21:54 2024 +1100 Update swiglu.py commit `f8aa20d2db` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 22:09:32 2024 +1100 Update fast_lora.py commit `ae0d9380c1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 22:07:10 2024 +1100 Update swiglu.py commit `694da2dba3` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 20:06:52 2024 +1100 Update fast_lora.py commit `4f573494da` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 20:04:51 2024 +1100 Update fast_lora.py commit `cb06ce849b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:41:48 2024 +1100 Update fast_lora.py commit `e0a36b356b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:30:18 2024 +1100 Update fast_lora.py commit `c4f0de58a9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:21:00 2024 +1100 Update fast_lora.py commit `a0a409d5af` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:20:53 2024 +1100 Update fast_lora.py commit `97658f9a74` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:12:08 2024 +1100 Update fast_lora.py commit `979de52220` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 19:00:22 2024 +1100 Update fast_lora.py commit `39d251757f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 18:19:36 2024 +1100 Update fast_lora.py commit `77365cc740` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 18:11:43 2024 +1100 Update llama.py commit `4bfbccec4b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 18:04:59 2024 +1100 Update fast_lora.py commit `56dffca23f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 17:44:10 2024 +1100 Update llama.py commit `259097bc61` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 17:36:40 2024 +1100 Update fast_lora.py commit `b8de6b6a9b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 17:25:36 2024 +1100 Update fast_lora.py commit `796aa4d0ef` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 17:13:09 2024 +1100 Update fast_lora.py commit `9ecb3bc869` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 17:04:33 2024 +1100 Update fast_lora.py commit `7bed11a8ba` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 16:43:50 2024 +1100 Update fast_lora.py commit `317bbc57fe` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 16:16:32 2024 +1100 Update fast_lora.py commit `c4ac728d4c` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 16:08:10 2024 +1100 Update fast_lora.py commit `ab44945336` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 14:06:25 2024 +1100 Update fast_lora.py commit `84772de40f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 13:59:57 2024 +1100 Update fast_lora.py commit `653f1036ae` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 03:49:48 2024 +1100 Update fast_lora.py commit `46ec8bbc3d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 03:15:23 2024 +1100 Repatch commit `99eeebf72a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 02:41:15 2024 +1100 Update swiglu.py commit `ae156d9154` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 02:07:35 2024 +1100 Update llama.py commit `29945bd8c1` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 26 01:33:23 2024 +1100 Update llama.py commit `194093e375` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 23:35:56 2024 +1100 remove patching commit `4f60055f9e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 23:25:17 2024 +1100 Update fast_lora.py commit `2d25facc7b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 23:14:37 2024 +1100 Update fast_lora.py commit `380f2fd6e4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 19:05:43 2024 +1100 Fix saving and bnb-4bit commit `0474451a6a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 03:55:26 2024 +1100 Update pyproject.toml commit `5a5d34b8ae` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 03:47:23 2024 +1100 Update mapper.py commit `bbf5ef65af` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 03:46:23 2024 +1100 Graceful FA2 error + torch 2.1.1 commit `5ea888c578` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Thu Jan 25 02:29:37 2024 +1100 Update to transformers 4.37 commit `d2f1521a49` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 23:23:05 2024 +1100 incorrect inference commit `f186fe9bc9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 19:44:45 2024 +1100 Update mistral.py commit `e184b06c4b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 19:28:17 2024 +1100 Update mistral.py commit `b314500993` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 19:23:58 2024 +1100 q_len issue commit `d6e85d7731` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 19:17:42 2024 +1100 q_len == 1 commit `5d9e68181e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 19:11:17 2024 +1100 hidden_states commit `f311a8efcc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 18:04:20 2024 +1100 Update llama.py commit `eeee6333ec` Merge: `20d8f22` `04f8771` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 18:04:10 2024 +1100 Merge branch 'main' into nightly commit `20d8f223d7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 03:47:28 2024 +1100 Fast LoRA saving commit `1bb1c3c2b9` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 03:36:50 2024 +1100 LoRA commit `24c7a67556` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:44:52 2024 +1100 Update llama.py commit `d87ef86991` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:22:37 2024 +1100 Update llama.py commit `a77c448939` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:18:37 2024 +1100 Update llama.py commit `5260ec2a0a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:13:01 2024 +1100 Update llama.py commit `716e03fe1b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:09:35 2024 +1100 Update llama.py commit `00f50876f7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:03:25 2024 +1100 Update llama.py commit `e5b5333137` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 02:00:10 2024 +1100 Update llama.py commit `9a5062e6c4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:44:10 2024 +1100 Update llama.py commit `1289ae825b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:41:20 2024 +1100 RoPE commit `7e4140ebfc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:37:36 2024 +1100 Update llama.py commit `1ba28d8e26` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:33:51 2024 +1100 Update llama.py commit `3e1b244d5e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:29:03 2024 +1100 Fast inference RoPE commit `8647f0e86e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 01:06:34 2024 +1100 Update llama.py commit `085a8e944a` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 23 00:32:57 2024 +1100 inference commit `f41a437540` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 22:34:33 2024 +1100 Update utils.py commit `31f0c9c08d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 22:31:38 2024 +1100 Update utils.py commit `4220f6a6dc` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 22:26:55 2024 +1100 No print commit `e1cbc9e423` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 22:21:29 2024 +1100 Update utils.py commit `050c61bc0b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 20:04:27 2024 +1100 Update utils.py commit `7c3b647ba7` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 20:01:59 2024 +1100 fast_linear_forward commit `8c613e851f` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 19:36:17 2024 +1100 Apache 2 commit `a18f982e67` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 19:31:34 2024 +1100 Max sequence lengths commit `e38485feaa` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 18:18:23 2024 +1100 Mistral correct RoPE scaling commit `278de9f375` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 04:33:44 2024 +1100 Update llama.py commit `0666589ace` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 04:28:24 2024 +1100 Update save.py commit `e34393f975` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 04:25:37 2024 +1100 Update llama.py commit `2a3d4f3a8d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 04:18:42 2024 +1100 fast inference commit `a5ab4dc21a` Merge: `8828ece` `3a9b2de` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Mon Jan 22 01:58:00 2024 +1100 Merge branch 'main' into nightly commit `8828eceb5b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 22:18:53 2024 +1100 Update llama.py commit `fcde58859b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 20:06:14 2024 +1100 Update llama.py commit `2f80890578` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 19:21:56 2024 +1100 Update llama.py commit `7be801ff8d` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 19:10:48 2024 +1100 Update llama.py commit `92512e670e` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 19:01:45 2024 +1100 Update llama.py commit `5c927e4930` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 19:01:28 2024 +1100 Update mistral.py commit `fe2bc30987` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 18:52:25 2024 +1100 Update llama.py commit `ac99a47a45` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 18:52:15 2024 +1100 Update llama.py commit `5bf108ebda` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 18:15:55 2024 +1100 Update llama.py commit `b591f33a37` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 18:06:44 2024 +1100 Update llama.py commit `da7b4f59ee` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 17:32:37 2024 +1100 Update save.py commit `196ab974d6` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 16:53:58 2024 +1100 Update llama.py commit `cbc1c69e29` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 16:22:28 2024 +1100 faster saving & inference	2024-03-17 22:21:36 +11:00
Daniel Han	64d847bede	Bug fixes (#257 ) * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md * Bugs * Update fast_lora.py * Update pyproject.toml * Update fast_lora.py	2024-03-17 22:09:50 +11:00
Daniel Han	c599ae0f27	Bug fixes (#249 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update rope_embedding.py * Update rope_embedding.py * Fix bugs * Update fast_lora.py * Update fast_lora.py * Update README.md * Update README.md * GGUF * Update save.py * Update save.py * Update save.py * Update save.py * Update README.md * Update README.md	2024-03-17 02:47:05 +11:00
Qubitium	39713e66ed	Fix single gpu limit code overriding the wrong cuda gpu id via env (#228 )	2024-03-16 00:12:16 +11:00
HuyNguyen-hust	e29a630cd3	10% faster RoPE embedding from HuyNguyen-hust (#238 )	2024-03-16 00:09:59 +11:00
Daniel Han	990c7a809c	Gemma GGUF chat templates work! (#246 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py	2024-03-15 05:09:45 +11:00
Daniel Han	2c5c5bb4bb	Fix Gemma GGUF (#234 ) * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py * Update save.py * GGUF incorrect * Update save.py * Update pyproject.toml * kaggle new * Update pyproject.toml * Update pyproject.toml * upcasting * Fix Colab * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml	2024-03-14 20:32:04 +11:00
Daniel Han	32223779c4	Fix more bugs (#232 ) * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update chat_templates.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Account for DoRA * Update llama.py	2024-03-11 04:31:03 +11:00
Daniel Han	8bea94c137	Saving fixes (#231 ) * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten * Update llama.py * Update llama.py * Update llama.py * Update save.py * Accuracy * Revert * Update save.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py	2024-03-10 20:09:34 +11:00
Daniel Han	1fcf9d4577	Fix bugs (#230 ) * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py * Update chat_templates.py * Update save.py * Update save.py * Update save.py * Update chat_templates.py * Update llama.py * model_name * Update loader.py * Tokenizer overwritten	2024-03-10 04:54:23 +11:00
Daniel Han	70f271b1d3	Fix Gemma (#223 ) * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Fix Gemma merging * Update rms_layernorm.py * Update gemma.py * Update pyproject.toml * Layernorms * Gemma precision * Update gemma.py * sqrt * Update gemma.py * Update save.py * RoPE and Gemma precision * Update rms_layernorm.py * Fix warning * Update chat_templates.py	2024-03-07 04:34:06 +11:00
Daniel Han	fedcafe281	Fix Gemma norm float32 (#217 ) * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py * Update rms_layernorm.py * Update rms_layernorm.py * Update rms_layernorm.py * Update gemma.py	2024-03-04 16:19:55 +11:00
Daniel Han	7b7665d9d6	Fix Gemma fast inference (#215 ) * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py * Update gemma.py	2024-03-03 19:36:06 +11:00
Daniel Han	fa2a43baf3	Fix Gemma activation function (#214 ) * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update pyproject.toml * Small fixes * Update pyproject.toml * Approx gelu * Update geglu.py * Approx gelu * Update llama.py * Update __init__.py * Update __init__.py * Update _utils.py * Update geglu.py	2024-03-03 18:21:44 +11:00
Daniel Han	307f2da353	Nightly (#204 ) * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py * Update llama.py * Hotfix - fix DoRA, Gemma prompt template (#202) (#203) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py	2024-02-29 00:18:38 +11:00
Daniel Han	25619645dd	Hotfix - fix DoRA, Gemma prompt template (#202 ) * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md * Update llama.py * DoRA * Update _utils.py * Update chat_templates.py	2024-02-29 00:15:22 +11:00
Daniel Han	e7c53fb370	2.4x faster Gemma (#197 ) * Update save.py * Update save.py * linking * llama.cpp bugs * Update save.py * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original * Gemma * Update pyproject.toml * Update mapper.py * Update fast_lora.py * FastGemmaModel * model_type * Update llama.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update fast_lora.py * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * gemma * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Fast CE Loss * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * CE * Update llama.py * Update llama.py * Update cross_entropy_loss.py * Update geglu.py * Update cross_entropy_loss.py * revert * Update llama.py * Update llama.py * norm * Update gemma.py * Update gemma.py * position_ids * Update gemma.py * Update gemma.py * pos * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * revert * revert * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * rope * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * llama * Update llama.py * gemma * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update gemma.py * Update save.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update gemma.py * correct_dtype * Update gemma.py * Update cross_entropy_loss.py * Update cross_entropy_loss.py * Chat Templates * Update README.md * Update README.md	2024-02-27 01:42:10 +11:00
Daniel Han	1b7bf718cc	Feb 2024 Release (#187 ) * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update save.py * Update save.py * Torch 2.2.0 * Update save.py * mistral swa * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Fix SWA inference * Fix llm_int8_skip_modules * SWA inference * Update save.py * Update save.py * Update pyproject.toml * __version__ * __version__ * Update save.py * Update save.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Chat Templates * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * patch tokenizer * Update chat_templates.py * Saving, LlamaRotaryEmbedding issues * Update llama.py * Update mistral.py * Update mapper.py * Fix RoPE precision issues * Bugs * saving bugs * Update llama.py * readme * spaces * spaces * globals * slash * slashes * spaces * apache * Update save.py * Update save.py * Update loader.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * trainer * Update save.py * Update pyproject.toml * install * Update save.py * Update save.py * Update save.py * Update save.py * PeftModel token + saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * linking * llama.cpp bugs * Update save.py * Update save.py * saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update __init__.py * Update save.py * Update save.py * Update save.py * save * trainer * spaces * original	2024-02-21 03:58:59 +11:00
Daniel Han	0439b8508d	Prelim Feb release (#173 ) * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update save.py * Update save.py * Torch 2.2.0 * Update save.py * mistral swa * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Fix SWA inference * Fix llm_int8_skip_modules * SWA inference * Update save.py * Update save.py * Update pyproject.toml * __version__ * __version__ * Update save.py * Update save.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Chat Templates * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * Update chat_templates.py * patch tokenizer * Update chat_templates.py * Saving, LlamaRotaryEmbedding issues * Update llama.py * Update mistral.py	2024-02-15 00:07:42 +11:00
Younes Belkada	474fd32f91	add HF tagging in unsloth (#170 )	2024-02-13 18:28:42 +11:00
Daniel Han	3d5cf373bc	Update README.md (#165 )	2024-02-09 15:59:17 +11:00
Daniel Han	2bc34566c4	Update README.md (#164 )	2024-02-09 15:49:09 +11:00
Daniel Han-Chen	99b8d231ce	Update mapper.py	2024-02-09 03:51:59 +11:00
Daniel Han	b7f24e804c	Update README.md (#162 )	2024-02-08 13:11:54 +11:00
Daniel Han	0b01dcb655	Nightly (#161 ) * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update save.py * Update save.py * Torch 2.2.0 * Update save.py * mistral swa * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Fix SWA inference * Fix llm_int8_skip_modules * SWA inference * Update save.py * Update save.py * Update pyproject.toml * __version__ * __version__ * Update save.py * Update save.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py	2024-02-08 03:40:28 +11:00
Daniel Han	25cfc7f590	Torch 2.2 (#157 ) * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update save.py * Update save.py * Torch 2.2.0 * Update save.py * mistral swa * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Fix SWA inference * Fix llm_int8_skip_modules * SWA inference * Update save.py * Update save.py * Update pyproject.toml * __version__ * __version__ * Update save.py * Update save.py * Update mistral.py	2024-02-07 04:40:50 +11:00
Michael Han	bb66faaa33	ReadMe Revamp (#156 ) * HF Perf Button * Update README.md Adding new buttons cleanup * Update README.md * Delete images/Discord.png * Delete images/try live demo green.png * new transparent logos * Revamping page * Revamp mainpage * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * finetune button * Delete start free finetune button.png * free finetune button * Add files via upload * Update README.md * Update README.md * Add files via upload * Add files via upload * Update README.md * Add files via upload * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Squashed commit of the following: commit `efa0d2332e` Author: Daniel Han <danielhanchen@gmail.com> Date: Sun Feb 4 17:35:56 2024 +1100 2x faster inference (#151) * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py commit `2f55935f94` Author: Daniel Han <danielhanchen@gmail.com> Date: Wed Jan 31 04:03:37 2024 +1100 Hotfix - fix inference (#146) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value commit `a3a2ad9382` Author: Daniel Han <danielhanchen@gmail.com> Date: Mon Jan 29 17:49:54 2024 +1100 Fix inference attention mask (#142) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py commit `90309ca8dc` Author: Daniel Han <danielhanchen@gmail.com> Date: Mon Jan 29 03:45:07 2024 +1100 Nightly (#140) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving commit `a16bc73e80` Author: Daniel Han <danielhanchen@gmail.com> Date: Mon Jan 29 02:52:39 2024 +1100 Fix saving issues (#139) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print commit `af33224554` Author: Daniel Han <danielhanchen@gmail.com> Date: Sun Jan 28 04:30:29 2024 +1100 1 more bug (#138) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py commit `e2bbd3819e` Author: Daniel Han <danielhanchen@gmail.com> Date: Sun Jan 28 04:20:06 2024 +1100 Fix bugs + more accurate Swiglu (#137) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask commit `a81aff286f` Author: Daniel Han <danielhanchen@gmail.com> Date: Sat Jan 27 04:50:22 2024 +1100 Inference bug fix (#134) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py commit `7da0c50f75` Author: Daniel Han <danielhanchen@gmail.com> Date: Sat Jan 27 04:47:54 2024 +1100 More bug fixes (#133) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py commit `62fae3aa74` Author: Daniel Han <danielhanchen@gmail.com> Date: Fri Jan 26 04:19:17 2024 +1100 Fix bugs (#129) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py commit `04f8771821` Author: Daniel Han <danielhanchen@gmail.com> Date: Tue Jan 23 03:55:24 2024 +1100 2-4x faster native HF inference (#119) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving commit `3a9b2dee98` Author: Daniel Han <danielhanchen@gmail.com> Date: Sun Jan 21 22:20:22 2024 +1100 Hotfix (#118) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py commit `a6f4fb0075` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 05:00:37 2024 +1100 Update save.py commit `705cac0357` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 04:21:54 2024 +1100 Update save.py commit `16edcb3be2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sun Jan 21 04:13:03 2024 +1100 Update save.py commit `3d05a74b12` Author: Daniel Han <danielhanchen@gmail.com> Date: Sun Jan 21 03:43:49 2024 +1100 Fixed saving! (#113) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Quick fixes * Update llama.py * Update llama.py * Update dpo.py * Update dpo.py * Update llama.py * Update save.py * getattr * RSLoRA and LoftQ direct support * Update llama.py * Update llama.py * Update llama.py * Fix DPO + GGUF * Fix quantization_method * Fix quantization_config * patch model * Update llama.py * Update llama.py * Update llama.py * Update save.py * Update save.py * tokenizer_save_settings * Update save.py * quantization and loftq * Update save.py * Update llama.py * Update save.py * upload_to_huggingface * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py commit `bb05d6b6e2` Author: Daniel Han <danielhanchen@gmail.com> Date: Sat Jan 20 23:23:00 2024 +1100 Hotfix for Jan 2024 Release (#110) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Quick fixes * Update llama.py * Update llama.py * Update dpo.py * Update dpo.py * Update llama.py * Update save.py * getattr * RSLoRA and LoftQ direct support * Update llama.py * Update llama.py * Update llama.py * Fix DPO + GGUF * Fix quantization_method * Fix quantization_config * patch model * Update llama.py * Update llama.py * Update llama.py * Update save.py * Update save.py * tokenizer_save_settings * Update save.py * quantization and loftq * Update save.py * Update llama.py * Update save.py commit `12e75c93d0` Author: Daniel Han <danielhanchen@gmail.com> Date: Sat Jan 20 04:25:06 2024 +1100 Quick fixes (#106) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Quick fixes * Update llama.py * Update llama.py * Update dpo.py * Update dpo.py * Update llama.py * Update save.py * getattr * RSLoRA and LoftQ direct support * Update llama.py * Update llama.py * Update llama.py * Fix DPO + GGUF commit `52b5ef31e0` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Sat Jan 20 02:30:31 2024 +1100 Update _utils.py commit `1a19c38675` Merge: `0a52390` `0d6e52b` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 19 23:15:38 2024 +1100 Merge branch 'main' of https://github.com/unslothai/unsloth commit `0a52390ac2` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 19 23:15:20 2024 +1100 Revert quantization methods commit `0d6e52b5c7` Author: Daniel Han <danielhanchen@gmail.com> Date: Fri Jan 19 22:57:22 2024 +1100 getattr issues (#103) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Quick fixes * Update llama.py * Update llama.py * Update dpo.py * Update dpo.py * Update llama.py * Update save.py * getattr commit `b3fcea6421` Author: Daniel Han <danielhanchen@gmail.com> Date: Fri Jan 19 22:52:30 2024 +1100 Quick fixes (#101) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Quick fixes * Update llama.py * Update llama.py * Update dpo.py * Update dpo.py * Update llama.py * Update save.py commit `d691516ab9` Author: Daniel Han <danielhanchen@gmail.com> Date: Fri Jan 19 04:51:19 2024 +1100 2024 Release (#96) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs * Faster saving + other changes * Update llama.py * Saving modules * spelling * Update llama.py * Update save.py * Update save.py * Update loader.py * Update llama.py * patch saving * Update save.py * Update save.py * Update save.py * patch saving * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * original_model * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * saving to RAM leakage? * Update save.py * new_save_directory * Update save.py * Update save.py * Update save.py * Update save.py * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml commit `9e2dec16fb` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 19 03:41:00 2024 +1100 Update pyproject.toml commit `396c7245dd` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Fri Jan 19 03:35:17 2024 +1100 Update pyproject.toml commit `738e91591f` Author: Daniel Han <danielhanchen@gmail.com> Date: Thu Jan 11 04:08:03 2024 +1100 Fix some bugs (#83) * Fix tokenizer, dropout, bias for LoRA * Update loader.py * Fix LoRA downcasting * Update _utils.py * Saving to GGUF * fix * colab_quantize_to_gguf * move save modules * save module * Update __init__.py * Update save.py * Temp downgrade due to TRL issue * Fix up bugs commit `a1da50b5ce` Author: Daniel Han <danielhanchen@gmail.com> Date: Wed Jan 10 23:10:48 2024 +1100 Update README.md (#81) commit `606e8a9284` Author: shimmy <107991372+shimmyshimmer@users.noreply.github.com> Date: Wed Jan 10 23:10:23 2024 +1100 Discord button redo (#80) commit `0169294ffb` Author: shimmy <107991372+shimmyshimmer@users.noreply.github.com> Date: Wed Jan 10 23:02:20 2024 +1100 Update logos (#79) * HF Perf Button * Update README.md Adding new buttons cleanup * Update README.md * Delete images/Discord.png * Delete images/try live demo green.png * new transparent logos * Revamping page * Revamp mainpage * Update README.md * Update README.md commit `b2a8c33430` Author: Daniel Han <danielhanchen@gmail.com> Date: Wed Jan 10 20:03:01 2024 +1100 Create FUNDING.yml (#78) commit `c9c1abf290` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Wed Jan 10 01:02:44 2024 +1100 fix_tokenizer commit `6efffb46e4` Author: Daniel Han-Chen <danielhanchen@gmail.com> Date: Tue Jan 9 23:40:43 2024 +1100 check_tokenizer --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>	2024-02-07 02:00:12 +11:00
Daniel Han	efa0d2332e	2x faster inference (#151 ) * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value * Update llama.py * Update llama.py * Fix SDPA * Update llama.py * padding * Inference * Update llama.py * Revert * Update mistral.py * faster inference * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * inference * Update llama.py * Update utils.py * faster inference * Update llama.py * revert * lm_head * Update llama.py * inference * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * faster inference * Update llama.py * fast inference * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * torch compile * past_key_values * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update utils.py * Update llama.py * fast inference + saving config.json * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * fast inference again * more temp matrices * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update mistral.py * Update llama.py * SDPA * attention_mask * New version * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update utils.py * Update utils.py	2024-02-04 17:35:56 +11:00
Daniel Han	2f55935f94	Hotfix - fix inference (#146 ) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py * Fast inference repatch * Update llama.py * Update utils.py * Update utils.py * Update utils.py * Update mistral.py * Update __init__.py * Fix inference * Update mistral.py * fast lm_head * Remove fast path * Update rope_embedding.py * Update loader.py * LlamaAttention_fast_forward_inference * if past_key_value is not None and q_len == 1: * revert inference * Update loader.py * past_key_value	2024-01-31 04:03:37 +11:00
Daniel Han	a3a2ad9382	Fix inference attention mask (#142 ) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving * Update llama.py * Update llama.py	2024-01-29 17:49:54 +11:00
Daniel Han	90309ca8dc	Nightly (#140 ) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print * Mistral patch * Update mistral.py * Update save.py * saving	2024-01-29 03:45:07 +11:00
Daniel Han	a16bc73e80	Fix saving issues (#139 ) * faster saving & inference * Update llama.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * fast inference * Update llama.py * Update save.py * Update llama.py * Mistral correct RoPE scaling * Max sequence lengths * Apache 2 * fast_linear_forward * Update utils.py * Update utils.py * No print * Update utils.py * Update utils.py * inference * Update llama.py * Fast inference RoPE * Update llama.py * Update llama.py * RoPE * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * LoRA * Fast LoRA saving * Update llama.py * hidden_states * q_len == 1 * q_len issue * Update mistral.py * Update mistral.py * incorrect inference * Update to transformers 4.37 * Graceful FA2 error + torch 2.1.1 * Update mapper.py * Update pyproject.toml * Fix saving and bnb-4bit * Update fast_lora.py * Update fast_lora.py * remove patching * Update llama.py * Update llama.py * Update swiglu.py * Repatch * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update llama.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update save.py * Update fast_lora.py * Update utils.py * Update llama.py * Update fast_lora.py * Update swiglu.py * Update save.py * Update save.py * Update llama.py * Update llama.py * Update llama.py * Update llama.py * Revert "Update llama.py" This reverts commit `a208ec46e0`. * Update llama.py * Works? * Update pyproject.toml * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Swiglu * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update swiglu.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * Update fast_lora.py * attention_mask * Update llama.py * Update llama.py * labels * Update mistral.py * Update llama.py * attention mask * Update save.py * Update save.py * Update mistral.py * attention mask * Update llama.py * Update llama.py * Update mistral.py * Update llama.py * Update llama.py * Update llama.py * Update dpo.py * Patch saving * Update save.py * Update save.py * patch_saving_functions * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * Update save.py * print	2024-01-29 02:52:39 +11:00

1 2 3 4

181 commits