From 6b95eccc2961eb47557efabd47507f4e94d6139d Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Wed, 29 May 2024 00:39:23 +1000 Subject: [PATCH] Updated Home (markdown) --- Home.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/Home.md b/Home.md index 2f6cd7e..3b73883 100644 --- a/Home.md +++ b/Home.md @@ -19,15 +19,14 @@ trainer.train() ``` ### Finetuning the `lm_head` and `embed_tokens` matrices: -Don't forget to resize your embedding matrices if you added new tokens! ```python model = FastLanguageModel.get_peft_model( model, r = 16, target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", - "gate_proj", "up_proj", "down_proj",], + "gate_proj", "up_proj", "down_proj", + "lm_head", "embed_tokens",], lora_alpha = 16, - modules_to_save = ["lm_head", "embed_tokens",], ) ``` @@ -128,11 +127,14 @@ model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16 ``` Then use the terminal and do: ```bash -git clone https://github.com/ggerganov/llama.cpp -cd llama.cpp && make clean && LLAMA_CUBLAS=1 make all -j +git clone --recursive https://github.com/ggerganov/llama.cpp +make clean -C llama.cpp +make all -j -C llama.cpp pip install gguf protobuf + +python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16 ``` -Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF. +Or follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF. ### Evaluation Loop - also OOM or crashing. Set the trainer settings for evaluation to: