Updated Home (markdown)

Daniel Han 2024-02-16 01:59:40 +11:00
commit fc25b4e525

19
Home.md

@ -55,16 +55,19 @@ model.save_pretrained_merged("model", tokenizer, save_method = "lora",)
model.push_to_hub_merged("hf/model", tokenizer, save_method = "lora", token = "")
```
### Evaluation Loop OOM or crashing.
### Evaluation Loop - also OOM or crashing.
Set the trainer settings for evaluation to:
```python
TrainingArguments(
fp16_full_eval = True,
per_device_eval_batch_size = 2,
eval_accumulation_steps = 4,
evaluation_strategy = "steps",
eval_steps = 1,
)
SFTTrainer(
args = TrainingArguments(
fp16_full_eval = True,
per_device_eval_batch_size = 2,
eval_accumulation_steps = 4,
evaluation_strategy = "steps",
eval_steps = 1,
),
train_dataset = train_dataset,
eval_dataset = eval_dataset,
```
This will cause no OOMs and make it somewhat faster with no upcasting to float32.