Updated Home (markdown)
parent
22e825789b
commit
6b95eccc29
1 changed files with 8 additions and 6 deletions
14
Home.md
14
Home.md
|
|
@ -19,15 +19,14 @@ trainer.train()
|
|||
```
|
||||
|
||||
### Finetuning the `lm_head` and `embed_tokens` matrices:
|
||||
Don't forget to resize your embedding matrices if you added new tokens!
|
||||
```python
|
||||
model = FastLanguageModel.get_peft_model(
|
||||
model,
|
||||
r = 16,
|
||||
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
|
||||
"gate_proj", "up_proj", "down_proj",],
|
||||
"gate_proj", "up_proj", "down_proj",
|
||||
"lm_head", "embed_tokens",],
|
||||
lora_alpha = 16,
|
||||
modules_to_save = ["lm_head", "embed_tokens",],
|
||||
)
|
||||
```
|
||||
|
||||
|
|
@ -128,11 +127,14 @@ model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16
|
|||
```
|
||||
Then use the terminal and do:
|
||||
```bash
|
||||
git clone https://github.com/ggerganov/llama.cpp
|
||||
cd llama.cpp && make clean && LLAMA_CUBLAS=1 make all -j
|
||||
git clone --recursive https://github.com/ggerganov/llama.cpp
|
||||
make clean -C llama.cpp
|
||||
make all -j -C llama.cpp
|
||||
pip install gguf protobuf
|
||||
|
||||
python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16
|
||||
```
|
||||
Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF.
|
||||
Or follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF.
|
||||
|
||||
### Evaluation Loop - also OOM or crashing.
|
||||
Set the trainer settings for evaluation to:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue