Updated Home (markdown)

Daniel Han 2024-05-29 00:39:23 +10:00
commit 6b95eccc29

14
Home.md

@ -19,15 +19,14 @@ trainer.train()
```
### Finetuning the `lm_head` and `embed_tokens` matrices:
Don't forget to resize your embedding matrices if you added new tokens!
```python
model = FastLanguageModel.get_peft_model(
model,
r = 16,
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",],
"gate_proj", "up_proj", "down_proj",
"lm_head", "embed_tokens",],
lora_alpha = 16,
modules_to_save = ["lm_head", "embed_tokens",],
)
```
@ -128,11 +127,14 @@ model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16
```
Then use the terminal and do:
```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUBLAS=1 make all -j
git clone --recursive https://github.com/ggerganov/llama.cpp
make clean -C llama.cpp
make all -j -C llama.cpp
pip install gguf protobuf
python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16
```
Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF.
Or follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF.
### Evaluation Loop - also OOM or crashing.
Set the trainer settings for evaluation to: