diff --git a/Home.md b/Home.md index 31b5e39..a97b1fe 100644 --- a/Home.md +++ b/Home.md @@ -116,13 +116,17 @@ ALLOWED_QUANTS = \ ``` ### Manually saving to GGUF -Firstly use the terminal and do: +First save your model to 16bit: +```python +model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16bit",) +``` +Then use the terminal and do: ```bash git clone https://github.com/ggerganov/llama.cpp cd llama.cpp && make clean && LLAMA_CUBLAS=1 make all -j pip install gguf protobuf ``` -Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model +Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF. ### Evaluation Loop - also OOM or crashing. Set the trainer settings for evaluation to: