ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Updated Home (markdown)

Daniel Han 2024-04-06 01:57:04 +11:00
commit 0641a982ab
1 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
Home.md

@ -116,13 +116,17 @@ ALLOWED_QUANTS = \
```
### Manually saving to GGUF
Firstly use the terminal and do:
First save your model to 16bit:
```python
model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16bit",)
```
Then use the terminal and do:
```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make clean && LLAMA_CUBLAS=1 make all -j
pip install gguf protobuf
```
Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model
Then follow the steps at https://rentry.org/llama-cpp-conversions#merging-loras-into-a-model using the model name "merged_model" to merge to GGUF.
### Evaluation Loop - also OOM or crashing.
Set the trainer settings for evaluation to: